Axonal stimulation affects the linear summation of single-point perception in three Argus II users.

Objective.Retinal implants use electrical stimulation to elicit perceived flashes of light ('phosphenes'). Single-electrode phosphene shape has been shown to vary systematically with stimulus parameters and the retinal location of the stimulating electrode, due to incidental activation of passing nerve fiber bundles. However, this knowledge has yet to be extended to paired-electrode stimulation.Approach.We retrospectively analyzed 3548 phosphene drawings made by three blind participants implanted with an Argus II Retinal Prosthesis. Phosphene shape (characterized by area, perimeter, major and minor axis length) and number of perceived phosphenes were averaged across trials and correlated with the corresponding single-electrode parameters. In addition, the number of phosphenes was correlated with stimulus amplitude and neuroanatomical parameters: electrode-retina and electrode-fovea distance as well as the electrode-electrode distance to ('between-axon') and along axon bundles ('along-axon'). Statistical analyses were conducted using linear regression and partial correlation analysis.Main results.Simple regression revealed that each paired-electrode shape descriptor could be predicted by the sum of the two corresponding single-electrode shape descriptors (p < .001). Multiple regression revealed that paired-electrode phosphene shape was primarily predicted by stimulus amplitude and electrode-fovea distance (p < .05). Interestingly, the number of elicited phosphenes tended to increase with between-axon distance (p < .05), but not with along-axon distance, in two out of three participants.Significance.The shape of phosphenes elicited by paired-electrode stimulation was well predicted by the shape of their corresponding single-electrode phosphenes, suggesting that two-point perception can be expressed as the linear summation of single-point perception. The impact of the between-axon distance on the perceived number of phosphenes provides further evidence in support of the axon map model for epiretinal stimulation. These findings contribute to the growing literature on phosphene perception and have important implications for the design of future retinal prostheses.

Regulation of band gap and localized surface plasmon resonance by loading Au nanorods on violet phosphene nanosheets for photodynamic/photothermal synergistic anti-infective therapy.

In the face of the serious threat to human health and the economic burden caused by bacterial antibiotic resistance, 2D phosphorus nanomaterials have been widely used as antibacterial agents. Violet phosphorus nanosheets (VPNSs) are an exciting bandgap-adjustable 2D nanomaterial due to their good physicochemical properties, yet the study of VPNS-based antibiotics is still in its infancy. Here, a composite of gold nanorods (AuNRs) loaded onto VPNS platforms (VPNS/AuNR) is constructed to maximize the potential of VPNSs for antimicrobial applications. The loading with AuNRs not only enhances the photothermal performance via a localized surface plasmon resonance (LSPR) effect, but also enhances the light absorption capacity due to the narrowing of the band gap of the VPNSs, thus increasing the ROS generation capacity. The results demonstrate that VPNS/AuNR exhibits outstanding antibacterial properties and good biocompatibility. Attractively, VPNS/AuNR is then extensively tested for treating skin wound infections, suggesting promising in vivo antibacterial and wound-healing features. Our findings may open a novel direction to develop a versatile VPNS-based treatment platform, which can significantly boost the progress of VPNS exploration.

Towards biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses.

Blindness affects millions of people around the world. A promising solution to restoring a form of vision for some individuals are cortical visual prostheses, which bypass part of the impaired visual pathway by converting camera input to electrical stimulation of the visual system. The artificially induced visual percept (a pattern of localized light flashes, or 'phosphenes') has limited resolution, and a great portion of the field's research is devoted to optimizing the efficacy, efficiency, and practical usefulness of the encoding of visual information. A commonly exploited method is non-invasive functional evaluation in sighted subjects or with computational models by using simulated prosthetic vision (SPV) pipelines. An important challenge in this approach is to balance enhanced perceptual realism, biologically plausibility, and real-time performance in the simulation of cortical prosthetic vision. We present a biologically plausible, PyTorch-based phosphene simulator that can run in real-time and uses differentiable operations to allow for gradient-based computational optimization of phosphene encoding models. The simulator integrates a wide range of clinical results with neurophysiological evidence in humans and non-human primates. The pipeline includes a model of the retinotopic organization and cortical magnification of the visual cortex. Moreover, the quantitative effects of stimulation parameters and temporal dynamics on phosphene characteristics are incorporated. Our results demonstrate the simulator's suitability for both computational applications such as end-to-end deep learning-based prosthetic vision optimization as well as behavioral experiments. The modular and open-source software provides a flexible simulation framework for computational, clinical, and behavioral neuroscientists working on visual neuroprosthetics.

Light flashes and other sensory illusions perceived in space travel and on ground, including proton and heavy ion therapies.

Most of the astronauts experience visual illusions, apparent flashes of light (LF) in absence of light. The first reported observation of this phenomenon was in July 1969 by Buzz Aldrin, in the debriefing following the Apollo 11 mission. Several ground-based experiments in the 1970s tried to clarify the mechanisms behind these light flashes and to evaluate possible related risks. These works were supported by dedicated experiments in space on the following Apollo flights and in Low Earth Orbit (LEO). It was soon demonstrated that the LF could be caused by charged particles (present in the space radiation) traveling through the eye, and, possibly, some other visual cortical areas. In the 1990s the interest in these phenomena increased again and additional experiments in Low Earth Orbit and others ground-based were started. Recently patients undergoing proton and heavy ion therapy for eye or head and neck tumors have reported the perception of light flashes, opening a new channel to investigate these phenomena. In this paper the many LF studies will be reviewed, presenting an historical and scientific perspective consistent with the combined set of observations, offering a single comprehensive summary aimed to provide further insights on these phenomena. While the light flashes appear not to be a risk by themselves, they might provide information on the amount of radiation induced radicals in the astronauts' eyes. Understanding their generation mechanisms might also support radiation countermeasures development. However, even given the substantial progress outlined in this paper, many questions related to their generation are still under debate, so additional studies are suggested. Finally, it is also conceivable that further LF investigations could provide evidence about the possible interaction of single particles in space with brain function, impacting with the crew ability to optimally perform a mission.

The Influence of Phosphene Synchrony in Driving Object Binding in a Simulation of Artificial Vision.

Purpose: Electrical microstimulation techniques used in visual prostheses are designed to restore visual function following acquired blindness. Patterns of induced focal percepts, known as phosphenes, are achieved by applying localized electrical pulses to the visual pathway to bypass the impaired site in order to convey images from the external world. Here, we use a simulation of artificial vision to manipulate relationships between individual phosphenes to observe the effects on object binding and perception. We hypothesize that synchronous phosphene presentation will facilitate object perception as compared to asynchronous presentation. Methods: A model system that tracks gaze position of normal, sighted participants to present patterns of phosphenes on a computer screen was used to simulate prosthetic vision. Participants performed a reading task at varying font sizes (1.1-1.4 logMAR) and under varying levels of phosphene temporal noise while reading accuracy and speed were measured. Results: Reading performance was significantly affected by temporal noise in phosphene presentation, with increasing desynchronization leading to lower reading scores. A drop in performance was also observed when the total latency between the gaze position and phosphene update was increased without adding temporal noise. Conclusions: Object perception (here, text perception) is enhanced with synchronously presented phosphenes as compared to asynchronously presented ones. These results are fundamental for developing an efficient temporal pattern of stimulation and for the creation of high-fidelity prosthetic vision.

Utilization of brain scans to create realistic phosphene maps for cortical visual prosthesis simulation studies.

It has been shown that we can restore sensations of light by stimulating the visual cortex. Cortical prosthetic vision consists of light perception in the visual field named phosphenes. Phosphenes are like pixels on a monitor which we can control to form the desired perception. However, the locations of phosphenes evoked vary between individuals. One of the biggest challenges is how to utilize phosphenes to present recognizable patterns that represent real-world scenes. Because of the difficulties of recruiting participants, and the risks of neurosurgery, researchers have used computer simulations to investigate the outcome of cortical visual prostheses. Previous simulations used regular phosphene maps, which may overestimate the visual ability cortical visual prosthesis can provide. This study aims to develop a more realistic simulation for cortical visual prostheses. We derived realistic phosphene maps using an existing cortical retinotopy dataset and decided implant placement by considering neurosurgery restrictions. We rendered some visual stimuli to evaluate the usability of those phosphene maps. The results indicate that presenting information on phosphenes maps may be more challenging than previously estimated.

A novel simulation paradigm utilising MRI-derived phosphene maps for cortical prosthetic vision.

Objective.We developed a realistic simulation paradigm for cortical prosthetic vision and investigated whether we can improve visual performance using a novel clustering algorithm.Approach.Cortical visual prostheses have been developed to restore sight by stimulating the visual cortex. To investigate the visual experience, previous studies have used uniform phosphene maps, which may not accurately capture generated phosphene map distributions of implant recipients. The current simulation paradigm was based on the Human Connectome Project retinotopy dataset and the placement of implants on the cortices from magnetic resonance imaging scans. Five unique retinotopic maps were derived using this method. To improve performance on these retinotopic maps, we enabled head scanning and a density-based clustering algorithm was then used to relocate centroids of visual stimuli. The impact of these improvements on visual detection performance was tested. Using spatially evenly distributed maps as a control, we recruited ten subjects and evaluated their performance across five sessions on the Berkeley Rudimentary Visual Acuity test and the object recognition task.Main results.Performance on control maps is significantly better than on retinotopic maps in both tasks. Both head scanning and the clustering algorithm showed the potential of improving visual ability across multiple sessions in the object recognition task.Significance.The current paradigm is the first that simulates the experience of cortical prosthetic vision based on brain scans and implant placement, which captures the spatial distribution of phosphenes more realistically. Utilisation of evenly distributed maps may overestimate the performance that visual prosthetics can restore. This simulation paradigm could be used in clinical practice when making plans for where best to implant cortical visual prostheses.

Abnormal visual and olfactory sensations during radiation therapy: a prospective study.

PURPOSE: Patients sometimes report phosphene and phantosmia during radiation therapy (RT). However, the detail features and related factors are not well understood. Our prospective study aimed to investigate the characteristics of phantosmias and phosphenes, to identify factors that influence the occurrence, intensity and hedonic (pleasantness/unpleasantness) ratings of such sensations during RT. METHODS: We included a total of 106 patients (37 women), who underwent RT in regions of the brain, ear, nose, throat (ENT), and other areas of the body for a duration of 43 ± 5 days. Medical history and treatment parameters were collected in a structured medical interview. Olfactory function was measured using the Sniffin' Stick Odor Identification Test at baseline. Phantosmia and phosphene were recorded weekly based on a self-report questionnaire. RESULTS: There were 37% of the patients experiencing phantosmias, 51% experiencing phosphenes, and 29% simultaneously experiencing both sensations. Phosphenes were typically perceived as a flashily blue, white and/or purple light, phantosmias were typically perceived as a chemical-like, metallic or burnt smell. Younger age (F = 7.81, p < 0.01), radiation in the brain region (χ2 = 14.05, p = 0.02), absence of taste problems (χ2 = 10.28, p = 0.01), and proton RT (χ2 = 10.57, p = 0.01) were related to these abnormal sensations. History of chemical/dust exposure predicted lower intensity (B = -1.52, p = 0.02) and lower unpleasantness (B = 0.49, p = 0.03) of phantosmia. In contrast, disease (tumor) duration (B = 0.11, p < 0.01), food allergy (B = 2.77, p < 0.01), and epilepsy (B = -1.50, p = 0.02) influence phosphenes intensity. Analgesics intake predicted a higher pleasantness of the phosphenes (B = 0.47, p < 0.01). CONCLUSIONS: Phantosmias and phosphenes are common during RT. The treatment settings and individual arousal level influence the occurrence, intensity and hedonic of such abnormal sensations. Phantosmias and phosphenes may involve more central neural than peripheral mechanism, and they could be elicited with activation of areas that are not regarded to be part of the olfactory or visual network.

Estimating Phosphene Locations Using Eye Movements of Suprachoroidal Retinal Prosthesis Users.

Purpose: Accurate mapping of phosphene locations from visual prostheses is vital to encode spatial information. This process may involve the subject pointing to evoked phosphene locations with their finger. Here, we demonstrate phosphene mapping for a retinal implant using eye movements and compare it with retinotopic electrode positions and previous results using conventional finger-based mapping. Methods: Three suprachoroidal retinal implant recipients (NCT03406416) indicated the spatial position of phosphenes. Electrodes were stimulated individually, and the subjects moved their finger (finger based) or their eyes (gaze based) to the perceived phosphene location. The distortion of the measured phosphene locations from the expected locations (retinotopic electrode locations) was characterized with Procrustes analysis. Results: The finger-based phosphene locations were compressed spatially relative to the expected locations all three subjects, but preserved the general retinotopic arrangement (scale factors ranged from 0.37 to 0.83). In two subjects, the gaze-based phosphene locations were similar to the expected locations (scale factors of 0.72 and 0.99). For the third subject, there was no apparent relationship between gaze-based phosphene locations and electrode locations (scale factor of 0.07). Conclusions: Gaze-based phosphene mapping was achievable in two of three tested retinal prosthesis subjects and their derived phosphene maps correlated well with the retinotopic electrode layout. A third subject could not produce a coherent gaze-based phosphene map, but this may have revealed that their phosphenes were indistinct spatially. Translational Relevance: Gaze-based phosphene mapping is a viable alternative to conventional finger-based mapping, but may not be suitable for all subjects.

Amplitude modulating frequency overrides carrier frequency in tACS-induced phosphene percept.

The amplitude modulated (AM) neural oscillation is an essential feature of neural dynamics to coordinate distant brain areas. The AM transcranial alternating current stimulation (tACS) has recently been adopted to examine various cognitive functions, but its neural mechanism remains unclear. The current study utilized the phosphene phenomenon to investigate whether, in an AM-tACS, the AM frequency could modulate or even override the carrier frequency in phosphene percept. We measured the phosphene threshold and the perceived flash rate/pattern from 12 human subjects (four females, aged from 20-44 years old) under tACS that paired carrier waves (10, 14, 18, 22 Hz) with different envelope conditions (0, 2, 4 Hz) over the mid-occipital and left facial areas. We also examined the phosphene source by adopting a high-density stimulation montage. Our results revealed that (1) phosphene threshold was higher for AM-tACS than sinusoidal tACS and demonstrated different carrier frequency functions in two stimulation montages. (2) AM-tACS slowed down the phosphene flashing and abolished the relation between the carrier frequency and flash percept in sinusoidal tACS. This effect was independent of the intensity change of the stimulation. (3) Left facial stimulation elicited phosphene in the upper-left visual field, while occipital stimulation elicited equally distributed phosphene. (4) The near-eye electrodermal activity (EDA) measured under the threshold-level occipital tACS was greater than the lowest power sufficient to elicit retinal phosphene. Our results show that AM frequency may override the carrier frequency and determine the perceived flashing frequency of AM-tACS-induced phosphene.

Clinical Progress and Optimization of Information Processing in Artificial Visual Prostheses.

Visual prostheses, used to assist in restoring functional vision to the visually impaired, convert captured external images into corresponding electrical stimulation patterns that are stimulated by implanted microelectrodes to induce phosphenes and eventually visual perception. Detecting and providing useful visual information to the prosthesis wearer under limited artificial vision has been an important concern in the field of visual prosthesis. Along with the development of prosthetic device design and stimulus encoding methods, researchers have explored the possibility of the application of computer vision by simulating visual perception under prosthetic vision. Effective image processing in computer vision is performed to optimize artificial visual information and improve the ability to restore various important visual functions in implant recipients, allowing them to better achieve their daily demands. This paper first reviews the recent clinical implantation of different types of visual prostheses, summarizes the artificial visual perception of implant recipients, and especially focuses on its irregularities, such as dropout and distorted phosphenes. Then, the important aspects of computer vision in the optimization of visual information processing are reviewed, and the possibilities and shortcomings of these solutions are discussed. Ultimately, the development direction and emphasis issues for improving the performance of visual prosthesis devices are summarized.

Percepts evoked by multi-electrode stimulation of human visual cortex.

BACKGROUND: Direct electrical stimulation of early visual cortex evokes the perception of small spots of light known as phosphenes. Previous studies have examined the location, size, and brightness of phosphenes evoked by stimulation of single electrodes. While it has been envisioned that concurrent stimulation of many electrodes could be used as the basis for a visual cortical prosthesis, the percepts resulting from multi-electrode stimulation have not been fully characterized. OBJECTIVE: To understand the rules governing perception of phosphenes evoked by multi-electrode stimulation of visual cortex. METHODS: Multi-electrode stimulation was conducted in human epilepsy patients. We examined the number and spatial arrangement of phosphenes evoked by stimulation of individual multi-electrode groups (n = 8), and the ability of subjects to discriminate between the pattern of phosphenes generated by stimulation of different multi-electrode groups (n = 7). RESULTS: Simultaneous stimulation of pairs of electrodes separated by greater than 4 mm tended to produce perception of two distinct phosphenes. Simultaneous stimulation of three electrodes gave rise to a consistent spatial pattern of phosphenes, but with significant variation in the absolute location, size, and orientation of that pattern perceived on each trial. Although multi-electrode stimulation did not produce perception of recognizable forms, subjects could use the pattern of phosphenes evoked by stimulation to perform simple discriminations. CONCLUSIONS: The number of phosphenes produced by multi-electrode stimulation can be predicted using a model for spread of activity in early visual cortex, but there are additional subtle effects that must be accounted for.

PVGAN: a generative adversarial network for object simplification in prosthetic vision.

Objective.By means of electrical stimulation of the visual system, visual prostheses provide promising solution for blind patients through partial restoration of their vision. Despite the great success achieved so far in this field, the limited resolution of the perceived vision using these devices hinders the ability of visual prostheses users to correctly recognize viewed objects. Accordingly, we propose a deep learning approach based on generative adversarial networks (GANs), termed prosthetic vision GAN (PVGAN), to enhance object recognition for the implanted patients by representing objects in the field of view based on a corresponding simplified clip art version.Approach.To assess the performance, an axon map model was used to simulate prosthetic vision in experiments involving normally-sighted participants. In these experiments, four types of image representation were examined. The first and second types comprised presenting phosphene simulation of real images containing the actual high-resolution object, and presenting phosphene simulation of the real image followed by the clip art image, respectively. The other two types were utilized to evaluate the performance in the case of electrode dropout, where the third type comprised presenting phosphene simulation of only clip art images without electrode dropout, while the fourth type involved clip art images with electrode dropout.Main results.The performance was measured through three evaluation metrics which are the accuracy of the participants in recognizing the objects, the time taken by the participants to correctly recognize the object, and the confidence level of the participants in the recognition process. Results demonstrate that representing the objects using clip art images generated by the PVGAN model results in a significant enhancement in the speed and confidence of the subjects in recognizing the objects.Significance.These results demonstrate the utility of using GANs in enhancing the quality of images perceived using prosthetic vision.

Reading text works better than watching videos to improve acuity in a simulation of artificial vision.

Simulated artificial vision is used in visual prosthesis design to answer questions about device usability. We previously reported a striking increase in equivalent visual acuity with daily use of a simulation of artificial vision in an active task, reading sentences, that required high levels of subject engagement, but passive activities are more likely to dominate post-implant experience. Here, we investigated the longitudinal effects of a passive task, watching videos. Eight subjects used a simulation of a thalamic visual prosthesis with 1000 phosphenes to watch 23 episodes of classic American television in daily, 25-min sessions, for a period of 1 month with interspersed reading tests that quantified reading accuracy and reading speed. For reading accuracy, we found similar dynamics to the early part of the learning process in our previous report, here leading to an improvement in visual acuity of 0.15 ± 0.05 logMAR. For reading speed, however, no change was apparent by the end of training. We found that single reading sessions drove about twice the improvement in acuity of single video sessions despite being only half as long. We conclude that while passive viewing tasks may prove useful for post-implant rehabilitation, active tasks are likely to be preferable.

Time stability and connectivity analysis with an intracortical 96-channel microelectrode array inserted in human visual cortex.

Objective.Microstimulation via electrodes that penetrate the visual cortex creates visual perceptions called phosphenes. Besides providing electrical stimulation to induce perceptions, each electrode can be used to record the brain signals from the cortex region under the electrode which contains brain state information. Since the future visual prosthesis interfaces will be implanted chronically in the visual cortex of blind people, it is important to study the long-term stability of the signals acquired from the electrodes. Here, we studied the changes over time and the repercussions of electrical stimulation on the brain signals acquired with an intracortical 96-channel microelectrode array implanted in the visual cortex of a blind volunteer for 6 months.Approach.We used variance, power spectral density, correlation, coherence, and phase coherence to study the brain signals acquired in resting condition before and after the administration of electrical stimulation during a period of 6 months.Main results.Variance and power spectral density up to 750 Hz do not show any significant trend in the 6 months, but correlation coherence and phase coherence significantly decrease over the implantation time and increase after electrical stimulation.Significance.The stability of variance and power spectral density in time is important for long-term clinical applications based on the intracortical signals collected by the electrodes. The decreasing trends of correlation, coherence, and phase coherence might be related to plasticity changes in the visual cortex due to electrical microstimulation.

Prediction of phosphenes occurrence in patients undergoing photon irradiation to ocular structures due to brain tumor.

BACKGROUND AND OBJECTIVE: Phosphenes are visual light phenomena that are experienced when there is no apparent light that stimulates the eye. In oncology, phosphenes are also present during radiation therapy for patients with tumors of the central nervous system, eyes, head and neck. Due to the discomfort of patients treated with irradiation to the head regions, research is conducted to determine whether the dose to the ocular structures is predictive for the occurrence of phosphenes. The objective was to demonstrate the relationship between the doses of the retina and vitreous humour with the appearance of phosphenes. MATERIAL AND METHOD: A descriptive study was carried out in a prospective cohort in 25 patients older than 18 years, with malignant tumours located at the level of the brain, both of primary and secondary origin, subjected to irradiation in 6 MV linear accelerators, during February 2020 to January 2021. As independent variables: Retinal dose and vitreous humour dose, and as dependent variables: Light flashes and stable light. Logistic regression analysis was used for prediction, using the SPSS statistical program (version 26.0). RESULT: A final date of 380 external radiotherapy treatments. The presence of any of the events in a prevalence of 58.7% of the total of fractions. The distribution for the presence of both events, flash of light and stable light, was 69.1%, 20.6% and 10.3% respectively. In the logistic regression analysis, for the light flare, only the dose factor in vitreous was significant (OR: 1.74, IC [1.059-2.419] p: 0.001). For stable light, the dose in the retina (OR: 1.73, IC [1.121-2.341] p: 0.005), and dose in the vitreous humor (OR: 1.82, IC [1.335-2.315] p: 0.003). CONCLUSIONS: There is a predictive relationship between the doses of irradiation of the retina and vitreous humour, for the generation of phosphenes. These results help radiotherapy centres take these anatomical structures into account to reduce the presence of phosphenes in patients. Likewise, it would help to reduce phosphenes, keeping the bunker area illuminated during the treatment.

Phosphene Attributes Depend on Frequency and Intensity of Retinal tACS.

Phosphene is the experience of light without natural visual stimulation. It can be induced by electrical stimulation of the retina, optic nerve or cortex. Induction of phosphenes can be potentially used in assistive devices for the blind. Analysis of phosphene might be beneficial for practical reasons such as adjustment of transcranial alternating current stimulation (tACS) frequency and intensity to eliminate phosphene perception (e.g., tACS studies using verum tACS group and sham group) or, on the contrary, to maximize perception of phosphenes in order to be more able to study their dynamics. In this study, subjective reports of 50 healthy subjects exposed to different intensities of retinal tACS at 4 different frequencies (6, 10, 20 and 40 Hz) were analyzed. The effectiveness of different tACS frequencies in inducing phosphenes was at least 92 %. Subject reported 41 different phosphene types; the most common were light flashes and light circles. Changing the intensity of stimulation often induced a change in phosphene attributes. Up to nine phosphene attributes changed when the tACS intensity was changed. Significant positive correlation was observed between number of a different phosphene types and tACS frequency. Based on these findings, it can be concluded that tACS is effective in eliciting phosphenes whose type and attributes change depending on the frequency and intensity of tACS. The presented results open new questions for future research.

A case study in phenomenology of visual experience with retinal prosthesis versus visual-to-auditory sensory substitution.

The phenomenology of the blind has provided an age-old, unparalleled means of exploring the enigmatic link between the brain and mind. This paper delves into the unique phenomenological experience of a man who became blind in adulthood. He subsequently underwent both an Argus II retinal prosthesis implant and training, and extensive training on the EyeMusic visual to auditory sensory substitution device (SSD), thereby becoming the first reported case to date of dual proficiency with both devices. He offers a firsthand account into what he considers the great potential of combining sensory substitution devices with visual prostheses as part of a complete visual restoration protocol. While the Argus II retinal prosthesis alone provided him with immediate visual percepts by way of electrically stimulated phosphenes elicited by the device, the EyeMusic SSD requires extensive training from the onset. Yet following the extensive training program with the EyeMusic sensory substitution device, our subject reports that the sensory substitution device allowed him to experience a richer, more complex perceptual experience, that felt more "second nature" to him, while the Argus II prosthesis (which also requires training) did not allow him to achieve the same levels of automaticity and transparency. Following long-term use of the EyeMusic SSD, our subject reported that visual percepts representing mainly, but not limited to, colors portrayed by the EyeMusic SSD are elicited in association with auditory stimuli, indicating the acquisition of a high level of automaticity. Finally, the case study indicates an additive benefit to the combination of both devices on the user's subjective phenomenological visual experience.

End-to-end optimization of prosthetic vision.

Neural prosthetics may provide a promising solution to restore visual perception in some forms of blindness. The restored prosthetic percept is rudimentary compared to normal vision and can be optimized with a variety of image preprocessing techniques to maximize relevant information transfer. Extracting the most useful features from a visual scene is a nontrivial task and optimal preprocessing choices strongly depend on the context. Despite rapid advancements in deep learning, research currently faces a difficult challenge in finding a general and automated preprocessing strategy that can be tailored to specific tasks or user requirements. In this paper, we present a novel deep learning approach that explicitly addresses this issue by optimizing the entire process of phosphene generation in an end-to-end fashion. The proposed model is based on a deep auto-encoder architecture and includes a highly adjustable simulation module of prosthetic vision. In computational validation experiments, we show that such an approach is able to automatically find a task-specific stimulation protocol. The results of these proof-of-principle experiments illustrate the potential of end-to-end optimization for prosthetic vision. The presented approach is highly modular and our approach could be extended to automated dynamic optimization of prosthetic vision for everyday tasks, given any specific constraints, accommodating individual requirements of the end-user.